Measuring Relative Performance of Accountable Care Organizations

Accountable Care Organizations (ACOs) represent healthcare delivery organizations which consist of groups of doctors, hospitals, and other health care providers, who come together voluntarily to provide coordinated, high quality care to patients that they serve. ACOs were first created under the tenet of the Patient Protection and Affordable Care Act (PPACA) and initially developed to provide high quality care to Medicare patients. Subsequently, their scope has expanded to include healthcare delivery to non-Medicare patients as well. Since the initial wave of ACOs have now been operational for a few years, it is important to study the determinants of ACO performance. In other words, what types of organizational factors are likely to be associated with superior ACO performance? Identification of such factors represents an important first step toward evaluating the types of managerial actions that can be undertaken in order to benchmark and improve under-performing ACOs.

In this research, we propose a sharper lens to study the impact of ACO organizational characteristics on ACO performance. Specifically, we propose a conceptualization of ACO relative performance, based on an ACO’s capability to use its mix of resource inputs to maximize its outputs. In the context of our study, inputs represent clinical and staff resources, as well as the number of inpatient and outpatient ER visits; while outputs represent different quality indicators of patient safety, preventive health, at-risk population and patient caregiver experience. Our approach draws on data envelopment analysis (DEA), a non-parametric approach for estimating relative efficiencies of decision-making units. Specifically, we seek to address two research questions: (a) what is the impact of ACO size on relative ACO performance, measured as ACOs ability to convert their clinical and non-clinical resources into quality outcomes, and (b) does health information technology moderate the impact of ACO size on performance?

We test our models using two years of a nationwide sample of ACO data for two years: 2014 and 2015, using a two-stage DEA and regression estimation approach. In the first stage, we compute the DEA efficiency scores using a BCC model; in the second stage, we then regress the DEA efficiency characterizations (where the dependent variable is binary) on ACO operational characteristics, using a logit estimation model. Our results indicate that larger ACOs, as measured by the number of assigned beneficiaries, are likely to exhibit lower performance levels relative to smaller ACOs, as measured by their ability to convert clinical and non-clinical resources into process quality outcomes.

However, we find that usage of electronic health records (EHR) mitigates the negative impact of ACO size. In other words, ACOs which use EHRs for patient care coordination are likely to realize greater performance compared to ACOs that do not use or adopt EHRs. Our models control for the effects of ACO type, patient risk, and location. Our results suggest that health policy makers should take a closer look at the impact of health IT as a potential platform to improve patient-provider care coordination as a means to realize greater performance in terms of health outcomes. Our results also indicate that larger is not always better when it comes to expanding their coverage in terms of covered beneficiaries. Our study also indicates that ACOs on the “advance payment” plan as well as those with greater operational experience are likely to exhibit greater efficiency, while patient risk is negatively correlated with ACO efficiency. Implications for healthcare researchers and recommendation for policy makers are discussed.